生而为人

程序员的自我修养

0%

部署任务

Posted on In

Auzre DevOps任务定时调度

步骤：Onboard AP App Deployment using AzDeployer (Stratus)

  1. Prepare workflow folder, master config and workflow config.
  2. Prepare APDrop.
  3. Create APGold virtual environment.
  4. Setup corresponding Azure Dev Ops CD pipeline. Add Azure DevOps release pipeline
1
2
3
4
##[warning]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\amd64\Microsoft.Common.CurrentVersion.targets(2203,5): Warning MSB3245: Could not resolve this reference. Could not locate the assembly "Google.Protobuf, Version=3.6.1.0, Culture=neutral, PublicKeyToken=a7d26565bac4d604, processorArchitecture=MSIL". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.


externalFeedCredentials: 'nuget-msazure-oldbond, nuget-msblox-azuregenevamonitoring, nuget-mscosmos-cosmosprod, nuget-mscosmos-cosmostest, nuget-msdata-bigdata, nuget-ossmsft-oss_all, nuget-trill-trill'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
2021-08-02 06:33:05 ThreadId:   69	PauseAndResumeWorkflows() Process: MSNBI_S2BHourly_0 UserCommand: Run
2021-08-02 06:33:05 ThreadId:   79	Getting information about event AdsBI_MSNBI_S2B_Hourly_Done
2021-08-02 06:33:05 ThreadId:   88	Getting information about event AdsBI_MSNBI_S2B_Hourly_Done
2021-08-02 06:33:05 ThreadId:   83	Getting information about event AdsBI_MSNBI_S2B_Hourly_Done
2021-08-02 06:33:05 ThreadId:   83	Got SQL exception: System.Data.SqlClient.SqlException (0x80131904): -999999:3:2--prc_ProcessStateGetNextDelta-212--Violation of PRIMARY KEY constraint 'PK_PROCESSPRESTATE'. Cannot insert duplicate key in object 'dbo.ProcessPrestate'. The duplicate key value is (-1931531102, 20210731 1100).
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
   at System.Data.SqlClient.SqlDataReader.get_MetaData()
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)
   at System.Data.SqlClient.SqlCommand.ExecuteReader()
   at Microsoft.AdCenter.WorkflowHost.CommunicationWorkflow.<>c__DisplayClass32_0.<GetLastParam>b__1(SqlConnection sqlConnection)
   at Microsoft.AdCenter.WorkflowHost.SqlConnectionHelper.RunQuery(Action`1 command)
ClientConnectionId:0fd2a26b-bb72-4bbd-baec-a16499dff840
Error Number:50000,State:1,Class:16
ClientConnectionId before routing:bfc0cfb4-53ac-4a8f-a985-44e2e512efe2
Routing Destination:b9d5d60a293f.tr147.westus2-a.worker.database.windows.net,11020
connection string: Server=adsdwctest.database.windows.net;Database=DWC_DB;User ID=AdsDataSI_Execution;Password=AnotherPassword8!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
2021-08-02T08:28:02.5629452Z          Checking compatibility for System.Security.Cryptography.Primitives 4.3.0 with .NETFramework,Version=v4.7.2.
2021-08-02T08:28:02.5630121Z          All packages and projects are compatible with .NETFramework,Version=v4.7.2.
2021-08-02T08:28:02.6040861Z          Committing restore...
2021-08-02T08:28:02.6041989Z          Generating MSBuild file D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\obj\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj.nuget.g.props.
2021-08-02T08:28:02.6048518Z          Generating MSBuild file D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\obj\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj.nuget.g.targets.
2021-08-02T08:28:02.6053133Z          Writing assets file to disk. Path: D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\obj\project.assets.json
2021-08-02T08:28:02.6174412Z          Writing cache file to disk. Path: D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\obj\project.nuget.cache
2021-08-02T08:28:02.6187683Z          Persisting dg to D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\obj\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj.nuget.dgspec.json
2021-08-02T08:28:02.6200559Z          Failed to restore D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj (in 17.33 sec).
2021-08-02T08:28:02.6260383Z          
2021-08-02T08:28:02.6264281Z          NuGet Config files used:
2021-08-02T08:28:02.6264975Z              D:\a\1\Nuget\tempNuGet_22994537.config
2021-08-02T08:28:02.6265449Z          
2021-08-02T08:28:02.6266040Z          Feeds used:
2021-08-02T08:28:02.6266664Z              https://msasg.pkgs.visualstudio.com/Shared%20Data/_packaging/Ads.BI.SubjectArea.Upstreams/nuget/v3/index.json
2021-08-02T08:28:02.6267321Z          
2021-08-02T08:28:02.6267938Z          Installed:
2021-08-02T08:28:02.6281713Z              60 package(s) to D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj
2021-08-02T08:28:02.6297867Z        Done executing task "RestoreTask" -- FAILED.
2021-08-02T08:28:02.6302830Z      1>Done building target "Restore" in project "dirs.proj" -- FAILED.
2021-08-02T08:28:02.6303710Z      1>Done Building Project "D:\a\1\s\private\src\Batch\dirs.proj" (Restore target(s)) -- FAILED.
2021-08-02T08:28:02.6405509Z 
2021-08-02T08:28:02.6547930Z Build FAILED.
2021-08-02T08:28:02.6562009Z 
2021-08-02T08:28:02.6580770Z        "D:\a\1\s\private\src\Batch\dirs.proj" (Restore target) (1) ->
2021-08-02T08:28:02.6589059Z        (Restore target) -> 
2021-08-02T08:28:02.6596250Z          D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605: Detected package downgrade: Microsoft.BI.Common from 4.21.0 to 4.5.0. Reference the package directly from the project to select a different version.  [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6599483Z        D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605:  Microsoft.BI.MSNMediation.HourlyS2B.Drop -> Ads.BI.StreamingToBatch 2.2.0 -> Microsoft.BI.Common (>= 4.21.0)  [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6602014Z        D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605:  Microsoft.BI.MSNMediation.HourlyS2B.Drop -> Microsoft.BI.Common (>= 4.5.0) [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6603542Z          D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605: Detected package downgrade: Microsoft.BI.Common from 4.12.0 to 4.5.0. Reference the package directly from the project to select a different version.  [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6605044Z        D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605:  Microsoft.BI.MSNMediation.HourlyS2B.Drop -> Ads.BI.Orchestration.Workflows 0.1.0 -> Microsoft.BI.Common (>= 4.12.0)  [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6606611Z        D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605:  Microsoft.BI.MSNMediation.HourlyS2B.Drop -> Microsoft.BI.Common (>= 4.5.0) [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6608307Z          D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605: Detected package downgrade: Microsoft.Bingads.Dwc.Tools from 2.2.0 to 2.0.5102033.20. Reference the package directly from the project to select a different version.  [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6609947Z        D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605:  Microsoft.BI.MSNMediation.HourlyS2B.Drop -> Ads.BI.Orchestration.Workflows 0.1.0 -> Microsoft.Bingads.Dwc.Tools (>= 2.2.0)  [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6612139Z        D:\a\1\s\private\src\Batch\SAMHourlyS2B\Microsoft.BI.MSNMediation.HourlyS2B.Drop\Microsoft.BI.MSNMediation.HourlyS2B.Drop.csproj : error NU1605:  Microsoft.BI.MSNMediation.HourlyS2B.Drop -> Microsoft.BingAds.Dwc.Tools (>= 2.0.5102033.20) [D:\a\1\s\private\src\Batch\dirs.proj]
2021-08-02T08:28:02.6612903Z 
2021-08-02T08:28:02.6613320Z     0 Warning(s)
2021-08-02T08:28:02.6613727Z     3 Error(s)
2021-08-02T08:28:02.6613981Z 
2021-08-02T08:28:02.6614474Z Time Elapsed 00:00:19.93
2021-08-02T08:28:02.7009246Z ##[error]Error: The process 'C:\Program Files\dotnet\dotnet.exe' failed with exit code 1
2021-08-02T08:28:02.7020201Z ##[error]Packages failed to restore









可知，package引入如果指定了低层级的版本，会强制上层package统一使用，类似java的原理。



    <PackageReference Include="Ads.BI.BinplacingTools">
      <Version>3.0.0</Version>
    </PackageReference>
    <PackageReference Include="Ads.BI.PipelineApps.Egress">
      <Version>6.1.0</Version>
    </PackageReference>
    <PackageReference Include="Microsoft.AdCenter.ExecuteSSISPackage">
      <Version>1.1.5102398-pacman</Version>
    </PackageReference>
    <PackageReference Include="Microsoft.Azure.KeyVault.Core">
      <Version>1.0.0</Version>
    </PackageReference>
    <PackageReference Include="Microsoft.BingAds.Dwc.Engine.Library">
      <Version>2.1.0</Version>
    </PackageReference>
    <PackageReference Include="Microsoft.Search.Autopilot">
      <Version>1.0.0</Version>
    </PackageReference>
    <PackageReference Include="WindowsAzure.Storage">
      <Version>7.1.2</Version>
    </PackageReference>

ads Data live-site tracking v2

Deployment process for FASTBI streaming pipeline - V2