(Regex) MatchCollection ohne For each schleife

  • VB.NET
  • .NET (FX) 4.5–4.8

Es gibt 6 Antworten in diesem Thema. Der letzte Beitrag () ist von Baa$.

    (Regex) MatchCollection ohne For each schleife

    Guten Tag,
    ich baue zurzeit ein Scraper und habe ein paar fragen.
    ich habe bereits Threading etc. eingebaut damit es alles im code schneller funktioniert, jedoch läuft mir alles zu langsam.

    VB.NET-Quellcode

    1. Public Sub ScrapeProxyDo(address As String)
    2. Dim wc As New Net.WebClient
    3. Dim matchCollection As MatchCollection
    4. Try
    5. Dim input As String = wc.DownloadString(address)
    6. matchCollection = REGEX.Matches(input)
    7. 'ncihts
    8. For Each obj As Object In matchCollection
    9. Dim match As Match = CType(obj, Match)
    10. Dim item As String = match.ToString()
    11. RichTextBox2.AppendText(item & Environment.NewLine)
    12. Next
    13. Catch ex As Exception
    14. 'Nichts
    15. End Try
    16. End Sub


    Der Code ist relativ simpel, er guckt ob die die seite eine IP mit einem Port beinhaltet die Regex dafür ist:

    VB.NET-Quellcode

    1. Dim REGEX As Regex = New Regex("\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\:[0-9]{1,5}\b")


    Nun jedoch nimmt er von dem string was runtergeladen wird jede proxy einzelt rauß und das dauert natürlich seine Zeit, kann man es vielleicht irgendwie abändern, dass er alle proxys rausfiltert und direkt in die RichTextBox einfügt?
    Das würde so nähmlich viel schneller gehen als wenn er sich von unten nach oben langsam durcharbeitet.

    LG

    Kashed schrieb:

    Nun jedoch nimmt er von dem string was runtergeladen wird jede proxy einzelt rauß und das dauert natürlich seine Zeit
    Ich glaub nicht, dass das gross was dauert.
    Ich verstehe aber auch nicht, was du meinst mit "proxis einzeln rausnehmen" im Gegensatz zu "alle proxys rausfiltert und direkt in die RichTextBox".

    Aber bau mal eine Stopwatch ein, und miss nach, wie lange das dauert.

    Randbemerkung:

    VB.NET-Quellcode

    1. For Each obj As Object In matchCollection
    2. Dim match As Match = CType(obj, Match)
    3. Dim item As String = match.ToString()
    4. RichTextBox2.AppendText(item & Environment.NewLine)
    5. Next

    Die beiden Variablen match und item, sind redundant.

    Besser, weil lesbarer , weil kürzer:

    VB.NET-Quellcode

    1. For Each obj As Match In matchCollection
    2. RichTextBox2.AppendText(match.ToString & Environment.NewLine)
    3. Next

    Vollzitat eines Vorposts durch Anredefunktion ersetzt ~VaporiZed

    @ErfinderDesRades:
    Guten Mittag,
    es dauert bei 100 Tausend linien ca. 6min und es soll in wenigen sekunden funktionieren,
    wenn sie den code ausprobieren, dann sehen sie was ich meine, dass jedes Item einzelt rausgenommen wird.
    Und ich will das die Regex komplett im String alle Matches rausfiltert und direkt einsetzt und sich nicht von unten nach Oben durcharbeitet.

    LG

    Dieser Beitrag wurde bereits 1 mal editiert, zuletzt von „VaporiZed“ ()

    Moin,
    gehe es etwas anders an schreib dir eine Scraper Funktion die eine List of string zurück gibt hier eine alte scraper source von mir.
    der war super schnell...

    VB.NET-Quellcode

    1. Imports System.Threading
    2. Imports System.IO
    3. Imports System.Net
    4. Imports System.Text
    5. Imports System.Text.RegularExpressions
    7. Public Class Form1
    8. Dim toolTip1 As New ToolTip()
    9. Private cts As CancellationTokenSource
    10. Private SourceList As New List(Of String), ScraperFound As New List(Of String)
    11. Dim WorkerBool As Boolean, SitesWithFinds As New List(Of String)
    12. Public SearchString As String = "[0-9]*?\.[0-9]*?\.[0-9]*?\.[0-9]*?\:[0-9]+", Remove_DP_Source As Boolean
    13. Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
    14. Progressbar_.Value = 0
    15. toolTip1.AutoPopDelay = 5000
    16. toolTip1.InitialDelay = 1000
    17. toolTip1.ReshowDelay = 500
    18. toolTip1.ShowAlways = True
    19. toolTip1.SetToolTip(Me.Start_Button, "Start")
    20. toolTip1.SetToolTip(Me.Load_Button, "Load")
    21. toolTip1.SetToolTip(Me.Clear_Button, "Clear")
    22. toolTip1.SetToolTip(Me.Save_Button, "Save")
    23. toolTip1.SetToolTip(Me.Settings_Button, "Settings")
    24. toolTip1.SetToolTip(Me.About_Button, "Developer")
    25. Button_Enable(True, False, False, True)
    26. Remove_DP_Source = True
    27. End Sub
    28. Private Sub Start_Button_Click(sender As Object, e As EventArgs) Handles Start_Button.Click
    29. If ListViewDB1.Items.Count <> 0 Then
    30. If Start_Button.ImageIndex = 5 Then toolTip1.SetToolTip(Me.Start_Button, "Start")
    31. 'If Start_Button.ImageIndex = 9 Then
    32. ' Me.Update_Status("Scraper Startet...")
    33. 'Else
    34. ' Me.Update_Status("Scraper Cancelled")
    35. 'End If
    36. If Start_Button.ImageIndex = 5 Then
    37. Start_Button.ImageIndex = 9
    38. If Start_Button.ImageIndex = 9 Then toolTip1.SetToolTip(Me.Start_Button, "Stop")
    39. cts = New CancellationTokenSource()
    40. Dim factory As New TaskFactory(cts.Token,
    41. TaskCreationOptions.PreferFairness,
    42. TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Current)
    43. Dim Progress As Integer = 0, Remaining As Integer = ListViewDB1.Items.Count
    44. Button_Enable(False, False, False, False)
    45. Me.Update_Status("Scraper Startet...")
    46. ScraperFound.Clear()
    47. SitesWithFinds.Clear()
    48. Progressbar_.Value = 0
    49. WorkerBool = True
    50. Try
    51. For Each iTEM As ListViewItem In ListViewDB1.Items
    52. factory.StartNew(New System.Action(Sub()
    53. If WorkerBool = True Then
    54. Dim List As New List(Of String)
    55. List = Me.SearchWebsiteForString(iTEM, Me.SearchString)
    56. If List.Count <> 0 Then
    57. SitesWithFinds.Add(iTEM.Text)
    58. Me.Update_Listview(iTEM, "Found: " & List.Count, Color.SeaGreen, Color.Lavender)
    59. Else
    60. Me.Update_Listview(iTEM, "Found: " & List.Count, Color.Crimson, Color.Lavender)
    61. End If
    62. Progress += 1 : Remaining -= 1
    63. Me.Update_Progressbar(Progress, ListViewDB1.Items.Count)
    64. 'Total Found: 0 | Remaining Sites: 0
    65. Me.Update_Status("Total Found: " & ScraperFound.Count & " | Remaining Sites: " & Remaining)
    66. Else
    67. cts.Cancel()
    68. Thread.Sleep(500)
    69. Me.Update_Status("Scraper Cancelled")
    70. End If
    71. End Sub))
    72. Next
    73. Task.WaitAny()
    74. Catch ex As OperationCanceledException
    75. Me.Update_Status("Scraper Cancelled")
    76. End Try
    77. Else
    78. WorkerBool = False
    79. Start_Button.ImageIndex = 5
    80. Button_Enable(True, True, True, True)
    81. End If
    82. Else
    83. MessageBox.Show("Pls, Load first Source List´s...")
    84. End If
    85. End Sub
    86. Private Sub Progressbar__ProgressChanged(sender As Object, Value As Integer) Handles Progressbar_.ProgressChanged
    87. If Progressbar_.Value = Progressbar_.Maximum Then
    88. Start_Button.ImageIndex = 5
    89. Me.Update_Status("Scraper Finished, with: " & ScraperFound.Count)
    90. MessageBox.Show("Scraper Finished, with: " & ScraperFound.Count)
    91. Button_Enable(True, True, True, True)
    92. End If
    93. End Sub
    94. Private Sub Button_Enable(ByVal Load As Boolean, Clear As Boolean, Save As Boolean, Settings As Boolean)
    95. Load_Button.Enabled = Load
    96. Clear_Button.Enabled = Clear
    97. Save_Button.Enabled = Save
    98. Settings_Button.Enabled = Settings
    99. End Sub
    100. Private Sub Load_Button_Click(sender As Object, e As EventArgs) Handles Load_Button.Click
    101. ListsLoader()
    102. End Sub
    103. Private Sub Clear_Button_Click(sender As Object, e As EventArgs) Handles Clear_Button.Click
    104. If SourceList.Count And ListViewDB1.Items.Count <> 0 Then
    105. SourceList.Clear()
    106. ScraperFound.Clear()
    107. ListViewDB1.Items.Clear()
    108. Me.Update_Status("All Cleared")
    109. Progressbar_.Value = 0
    110. Else
    111. End If
    112. Button_Enable(True, False, False, True)
    113. End Sub
    114. Private Sub ListsLoader()
    115. SourceList.Clear()
    116. Dim InterneList As New List(Of String)
    117. Dim ofd As New OpenFileDialog With {
    118. .Title = "Select Proxie Scraping List",
    119. .Filter = "TXT (*.txt)|*.txt",
    120. .FilterIndex = 0,
    121. .InitialDirectory = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments),
    122. .Multiselect = True}
    123. If ofd.ShowDialog = Windows.Forms.DialogResult.OK Then
    124. For Each filename As String In ofd.FileNames
    125. InterneList.AddRange(IO.File.ReadAllLines(filename))
    126. Next
    127. SourceList = Me.RemoveDuplicate(InterneList)
    128. If MessageBox.Show("Remove " & InterneList.Count - SourceList.Count & " Duplicated Links?", "Frage", MessageBoxButtons.YesNo) _
    129. = Windows.Forms.DialogResult.Yes Then
    130. For Each Link In SourceList
    131. ListViewDB1.Items.Add(Link)
    132. Next
    133. Else
    134. For Each Link In InterneList
    135. ListViewDB1.Items.Add(Link)
    136. Next
    137. End If
    138. Button_Enable(False, True, False, True)
    139. Else
    140. Exit Sub
    141. End If
    142. Me.Update_Status("Addet " & ListViewDB1.Items.Count & " Source Links to List..")
    143. End Sub
    144. Private Sub Save_Button_Click(sender As Object, e As EventArgs) Handles Save_Button.Click
    145. If ScraperFound.Count <> 0 Then
    146. Dim s As New SaveFileDialog
    147. s.Filter = "text|*.txt"
    148. s.Title = "Save All Finds!"
    149. If s.ShowDialog = DialogResult.OK Then
    150. Dim fileName = s.FileName
    151. Dim sb As New StringBuilder
    152. For Each Proxy As String In ScraperFound
    153. sb.AppendLine($"{Proxy}")
    154. Next
    155. File.WriteAllText(fileName, sb.ToString)
    156. End If
    157. Status_Label.Text = "Proxys Saved!"
    158. Else
    159. Exit Sub
    160. End If
    161. End Sub
    162. Private Sub Settings_Button_Click(sender As Object, e As EventArgs) Handles Settings_Button.Click
    163. Setform.Show()
    164. End Sub
    165. Private Sub About_Button_Click(sender As Object, e As EventArgs) Handles About_Button.Click
    166. MessageBox.Show("Hi i Call myself DevilEggs" & vbCrLf & "and am a German Programmer, Designer etc." & vbCrLf & "Greetz to All boyz in the Hood BK-CRIME591 & EINHEIT591")
    167. End Sub
    168. Private Sub SaveEveryoneWithFindsToolStripMenuItem_Click(sender As Object, e As EventArgs) Handles SaveEveryoneWithFindsToolStripMenuItem.Click
    169. If ScraperFound.Count <> 0 Then
    170. Dim s As New SaveFileDialog
    171. s.Filter = "text|*.txt"
    172. s.Title = "Save All Links with Finds!"
    173. If s.ShowDialog = DialogResult.OK Then
    174. Dim fileName = s.FileName
    175. Dim sb As New StringBuilder
    176. For Each Proxy As String In SitesWithFinds
    177. sb.AppendLine($"{Proxy}")
    178. Next
    179. File.WriteAllText(fileName, sb.ToString)
    180. End If
    181. Status_Label.Text = "Saved!"
    182. Else
    183. Exit Sub
    184. End If
    185. End Sub
    186. #Region "function"
    187. Function RemoveDuplicate(ByVal TheList As List(Of String)) As List(Of String)
    188. Dim Result As New List(Of String)
    189. Dim Exist As Boolean
    190. For Each ElementString As String In TheList
    191. Exist = False
    192. For Each ElementStringInResult As String In Result
    193. If ElementString = ElementStringInResult Then
    194. Exist = True
    195. Exit For
    196. End If
    197. Next
    198. If Not Exist Then
    199. Result.Add(ElementString)
    200. End If
    201. Next
    202. Return Result
    203. End Function
    204. Public Function SearchWebsiteForString(ByVal Url As ListViewItem, ByVal Regex As String) As List(Of String)
    205. If Equals(Url, Nothing) Or Equals(Regex, Nothing) Then Throw New ArgumentNullException(Url.SubItems(1).Text)
    206. Dim result As New List(Of String)
    207. Dim Status As Integer
    208. Try
    209. If Url.Text = Nothing Then
    210. Exit Function
    211. Else
    212. Dim webRequest = CType(HttpWebRequest.Create(Url.Text), HttpWebRequest)
    213. Using webResponse As HttpWebResponse = webRequest.GetResponse()
    214. webRequest.Timeout = 1600
    215. webRequest.AllowAutoRedirect = True
    216. Status = webResponse.StatusCode
    217. Using SR As StreamReader = New StreamReader(webResponse.GetResponseStream())
    218. Dim content As String = SR.ReadToEnd
    219. Dim r As Regex = New Regex(Regex, RegexOptions.IgnoreCase)
    220. Dim match As Match = r.Match(content)
    221. While match.Success
    222. result.Add(match.Value)
    223. ScraperFound.Add(match.Value)
    224. match = match.NextMatch()
    225. End While
    226. End Using
    227. End Using
    228. End If
    229. Catch ex As Exception
    231. End Try
    232. Return result
    233. End Function
    234. #End Region
    235. #Region "invoke subs"
    236. Private Sub Update_Listview(ByVal iTEM As ListViewItem, ByVal Nachricht As String, ByVal Forer As Color, ByVal Back As Color)
    237. Dim ListviewItemSub As New ListViewItem.ListViewSubItem
    238. ListviewItemSub.Text = Nachricht
    239. ListviewItemSub.ForeColor = Forer
    240. ListviewItemSub.BackColor = Back
    241. Try
    242. If iTEM.SubItems(1).Text.Length <> 0 Then
    243. iTEM.SubItems.Remove(iTEM.SubItems(1))
    244. End If
    246. Catch ex As Exception
    247. End Try
    248. Try
    249. If Me.ListViewDB1.InvokeRequired Then
    250. Me.ListViewDB1.Invoke(Sub() Me.ListViewDB1.Items.Item(iTEM.Index).UseItemStyleForSubItems = False)
    251. Me.ListViewDB1.Invoke(Sub() Me.ListViewDB1.Items.Item(iTEM.Index).SubItems.Add(ListviewItemSub))
    252. Else
    253. Me.ListViewDB1.Items.Item(iTEM.Index).UseItemStyleForSubItems = False
    254. Me.ListViewDB1.Items.Item(iTEM.Index).SubItems.Add(ListviewItemSub)
    255. End If
    256. Catch ex As Exception
    258. End Try
    259. End Sub
    260. Private Sub Update_Status(ByVal News As String)
    261. If Me.Status_Label.InvokeRequired Then
    262. Me.Status_Label.Invoke(Sub() Me.Status_Label.Text = News)
    263. Else
    264. Me.Status_Label.Text = News
    265. End If
    266. End Sub
    267. Private Sub Update_Progressbar(paramValue As Integer, paramMaximum As Integer)
    268. If Me.Progressbar_.InvokeRequired Then
    269. Me.Progressbar_.Invoke(Sub() Me.Update_Progressbar(paramValue, paramMaximum))
    270. Else
    271. Me.Progressbar_.Maximum = paramMaximum
    272. Me.Progressbar_.Value = paramValue
    273. Me.Progressbar_.Update()
    274. End If
    275. End Sub
    276. #End Region
    277. End Class

    Dieser Beitrag wurde bereits 1 mal editiert, zuletzt von „Baa$“ ()

Facebook
Telefonspende